Can You See the (Linguistic) Difference? Exploring Mass/Count Distinction in Vision

This work explores the linguistic distinction between count and mass nouns in the visual modality. Since the former class typically refers to well-defined, countable objects, with the latter prototypically including less countable substances, we explore to which extent the linguistic distinction is grounded in the visual representations of the entities denoted by count/mass nouns. Using visual features extracted from a state-of-the-art Convolutional Neural Network (CNN), we show that the entities referred to as mass exhibit a lower variance both internally (i.e. intra-image) and externally (i.e. inter-image) compared to count. That is, various instances of substances are internally more homogeneous and externally more consistent to each other than are count. We compare variance across various CNN layers and show that it is indicative of the categorization when low-level features of the images are used, whereas any effect disappears when experimenting with higher-level, more abstract representations.